Exploratory Analysis and Dashboards

---

Milestone 3: Exploratory Analysis and Dashboard Submission

Once we have the data pre-processed, we are ready to roll up our sleeves and begin exploring the data in earnest. In this milestone you will use the skills that you have learned in the specialization to perform exploratory analysis of your data. You will identify key metrics in the data and create KPIs to track those metrics. We will use those KPIs to create dashboards that allow for comparative views and “brushing and linking” of the data. This will allow us to begin to think about the proper context of developing an explanatory analysis that will form the basis for the remaining milestones.

Remember that at this point, your design decisions should be made to facilitate exploration of the data, so be sure your KPIs and worksheets demonstrate the visual and cognitive design principles learned in throughout the Specialization. In particular, and make use of advanced features like hierarchies, actions, filters and parameters.

You will submit a URL to your worksheets or dashboard on Tableau Public containing your KPIs (key metrics) for this project. Your worksheets or dashboard must make use of at least one design principle and at least one KPI principle taught in this course. You will also submit a written narrative answering specific questions.

Import Libraries

import numpy as np
from numpy import count_nonzero, median, mean
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import plotly.express as px
import random

%matplotlib inline
#sets the default autosave frequency in seconds
%autosave 60 
sns.set_style('dark')
sns.set(font_scale=1.2)

plt.rc('axes', titlesize=9)
plt.rc('axes', labelsize=14)
plt.rc('xtick', labelsize=12)
plt.rc('ytick', labelsize=12)

import warnings
warnings.filterwarnings('ignore')

pd.set_option('display.max_columns',None)
#pd.set_option('display.max_rows',None)
pd.set_option('display.width', 1000)
pd.set_option('display.float_format','{:.2f}'.format)

random.seed(0)
np.random.seed(0)
np.set_printoptions(suppress=True)

Autosaving every 60 seconds

Data Visualization

df = pd.read_csv("clean.csv")
df.head()

	date	quarter	department	day	team	targeted_productivity	smv	wip	over_time	incentive	idle_time	idle_men	no_of_style_change	no_of_workers	actual_productivity	prod_diff
0	2015-01-01	Quarter1	sweing	Thursday	8	0.80	26.16	1108.00	6960.00	50.00	0.00	0	0	59.00	0.94	0.14
1	2015-01-01	Quarter1	finishing	Thursday	1	0.75	3.94	1190.47	960.00	0.00	0.00	0	0	8.00	0.89	0.14
2	2015-01-01	Quarter1	sweing	Thursday	11	0.80	11.41	968.00	3660.00	50.00	0.00	0	0	30.50	0.80	0.00
3	2015-01-01	Quarter1	sweing	Thursday	12	0.80	11.41	968.00	3660.00	50.00	0.00	0	0	30.50	0.80	0.00
4	2015-01-01	Quarter1	sweing	Thursday	6	0.80	25.90	1170.00	1920.00	50.00	0.00	0	0	56.00	0.80	0.00

Groupby Function

pd.DataFrame(df.groupby('quarter')['actual_productivity'].mean())

	actual_productivity
quarter
Quarter1	0.75
Quarter2	0.74
Quarter3	0.70
Quarter4	0.71
Quarter5	0.83

pd.DataFrame(df.groupby('department')['actual_productivity'].mean())

	actual_productivity
department
finishing	0.75
sweing	0.72

pd.DataFrame(df.groupby('day')['actual_productivity'].mean())

	actual_productivity
day
Monday	0.74
Saturday	0.75
Sunday	0.73
Thursday	0.72
Tuesday	0.74
Wednesday	0.73

pd.DataFrame(df.groupby('team')['actual_productivity'].mean())

	actual_productivity
team
1	0.82
2	0.77
3	0.80
4	0.77
5	0.70
6	0.69
7	0.67
8	0.67
9	0.73
10	0.72
11	0.68
12	0.78

Univariate Data Exploration

# Plot 4 rows and 1 column (can be expanded)

fig, ax = plt.subplots(4,1, sharex=False, sharey=False, figsize=(16,20))
#fig.suptitle('Main Title')


sns.scatterplot(x="smv", y="actual_productivity", data=df, ax=ax[0], ci=None, sizes=20)
ax[0].set_title('Title of the first chart')
#ax[0].tick_params('x', labelrotation=45)
ax[0].axhline(y = 0.8, color = 'r', linestyle = '-')
ax[0].set_xlabel("SMV")
ax[0].set_ylabel("")

sns.scatterplot(x="wip", y="actual_productivity", data=df, ax=ax[1], ci=None, sizes=20)
ax[1].set_title('Title of the second chart')
#ax[1].tick_params('x', labelrotation=45)
ax[1].axhline(y = 0.8, color = 'r', linestyle = '-')
ax[1].set_xlabel("WIP")
ax[1].set_ylabel("")

sns.scatterplot(x="over_time", y="actual_productivity", data=df, ax=ax[2], ci=None, sizes=20)
ax[2].set_title('Title of the third chart')
#ax[2].tick_params('x', labelrotation=45)
ax[2].axhline(y = 0.8, color = 'r', linestyle = '-')
ax[2].set_xlabel("Overtime")
ax[2].set_ylabel("")

sns.scatterplot(x="incentive", y="actual_productivity", data=df, ax=ax[3], ci=None, sizes=20)
ax[3].set_title('Title of the fourth chart')
#ax[3].tick_params('x', labelrotation=45)
ax[3].axhline(y = 0.8, color = 'r', linestyle = '-')
ax[3].set_xlabel("Incentive")
ax[3].set_ylabel("")

plt.tight_layout()
plt.show()

df.columns

Index(['date', 'quarter', 'department', 'day', 'team', 'targeted_productivity', 'smv', 'wip', 'over_time', 'incentive', 'idle_time', 'idle_men', 'no_of_style_change', 'no_of_workers', 'actual_productivity', 'prod_diff'], dtype='object')

fig = px.scatter(data_frame=df, x="incentive", y="actual_productivity", 
                 title="Incentive vs Productivity", 
                 labels=dict(actual_productivity="Actual Productivity", incentive="Incentive"))
fig.show()

# Plot 4 rows and 1 column (can be expanded)

fig, ax = plt.subplots(4,1, sharex=False, sharey=False, figsize=(16,16))
#fig.suptitle('Main Title')


sns.scatterplot(x="idle_time", y="actual_productivity", data=df, ax=ax[0], ci=None, sizes=20)
ax[0].set_title('Title of the first chart')
#ax[0].tick_params('x', labelrotation=45)
ax[0].axhline(y = 0.8, color = 'r', linestyle = '-')
ax[0].set_xlabel("Idle Time")
ax[0].set_ylabel("")

sns.scatterplot(x="idle_men", y="actual_productivity", data=df, ax=ax[1], ci=None, sizes=20)
ax[1].set_title('Title of the second chart')
#ax[1].tick_params('x', labelrotation=45)
ax[1].axhline(y = 0.8, color = 'r', linestyle = '-')
ax[1].set_xlabel("Idle Men")
ax[1].set_ylabel("")

sns.scatterplot(x="no_of_style_change", y="actual_productivity", data=df, ax=ax[2], ci=None, sizes=20)
ax[2].set_title('Title of the third chart')
#ax[2].tick_params('x', labelrotation=45)
ax[1].axhline(y = 0.8, color = 'r', linestyle = '-')
ax[2].set_xlabel("No of Style Change")
ax[2].set_ylabel("")

sns.scatterplot(x="no_of_workers", y="actual_productivity", data=df, ax=ax[3], ci=None, sizes=20)
ax[3].set_title('Title of the fourth chart')
#ax[3].tick_params('x', labelrotation=45)
ax[3].axhline(y = 0.8, color = 'r', linestyle = '-')
ax[3].set_xlabel("No of Workers")
ax[3].set_ylabel("")

plt.tight_layout()
plt.show()

fig = px.scatter(data_frame=df, x="no_of_workers", y="actual_productivity", 
                 title="No of Workers vs Productivity", 
                 labels=dict(actual_productivity="Actual Productivity", no_of_workers="No of Workers"))
fig.show()

g = sns.catplot(x='actual_productivity',  col = 'quarter',
            kind='box', data=df,
            height = 5, aspect = 1)

g.set_xlabels("Productivity")
g.set_ylabels("")
#g = (g.set_axis_labels("Tip","Total bill(USD)").set(xlim=(0,10),ylim=(0,100)


g.set(xlim=(0,None))
g.set_xticklabels(rotation=90)

plt.suptitle('Actual Productivity Quarterly', x=0.5, y=1.02, ha='center', fontsize=20)
plt.tight_layout()
plt.show()

fig = px.box(data_frame=df, y="actual_productivity", facet_col="quarter", color="quarter",
             title="Actual Productivity Quarterly", labels=dict(actual_productivity='Actual Productivity'))
fig.show()

g = sns.catplot(x='actual_productivity',  col = 'department',
            kind='box', data=df,
            height = 5, aspect = 1)

g.set_xlabels("Productivity")
g.set_ylabels("")
#g = (g.set_axis_labels("Tip","Total bill(USD)").set(xlim=(0,10),ylim=(0,100)


g.set(xlim=(0,None))
g.set_xticklabels(rotation=90)

plt.suptitle('Actual Productivity Dept', x=0.5, y=1.02, ha='center', fontsize=20)
plt.tight_layout()
plt.show()

fig = px.box(data_frame=df, y="actual_productivity", facet_col="department", color="department",
             title="Actual Productivity Department", labels=dict(actual_productivity='Actual Productivity'))
fig.show()

g = sns.catplot(x='actual_productivity',  col = 'day',
            kind='box', data=df, col_wrap=3,
            height = 5, aspect = 1)

g.set_xlabels("Productivity")
g.set_ylabels("")
#g = (g.set_axis_labels("Tip","Total bill(USD)").set(xlim=(0,10),ylim=(0,100)


g.set(xlim=(0,None))
g.set_xticklabels(rotation=90)

plt.suptitle('Actual Productivity Daily', x=0.5, y=1.02, ha='center', fontsize=20)
plt.tight_layout()
plt.show()

fig = px.box(data_frame=df, y="actual_productivity", facet_col="day", color="day",
             title="Actual Productivity by Day", labels=dict(actual_productivity='Actual Productivity'))
fig.show()

g = sns.catplot(x='actual_productivity',  col = 'team',
            kind='box', data=df, col_wrap=4,
            height = 5, aspect = 1)

g.set_xlabels("Productivity")
g.set_ylabels("")
#g = (g.set_axis_labels("Tip","Total bill(USD)").set(xlim=(0,10),ylim=(0,100)


g.set(xlim=(0,None))
g.set_xticklabels(rotation=90)

plt.suptitle('Actual Productivity by Team', x=0.5, y=1.02, ha='center', fontsize=20)
plt.tight_layout()
plt.show()

fig = px.box(data_frame=df, y="actual_productivity", facet_col="team", color="team", facet_col_wrap=4, height=1200,
             title="Actual Productivity by Team", labels=dict(actual_productivity='Actual Productivity'))
fig.show()

Time-Series Analysis

fig = plt.figure(figsize=(30,10))
sns.lineplot(x=df.date,y=df.actual_productivity, data=df, estimator=None)
plt.title("Actual Productivity in Jan-Mar 2015", fontsize=20)
plt.xlabel("Dates", fontsize=20)
plt.axhline(y = 0.8, color = 'r', linestyle = '-')
plt.xticks(rotation=90)
plt.ylabel("", fontsize=20)
#plt.legend(['',''])
plt.tight_layout()
plt.show()

fig = px.line(data_frame=df, x="date", y="actual_productivity", title="Actual Productivity in Jan-Mar 2015",
             labels={'actual_productivity': 'Actual Productivity','date':'Date'})

fig.show()

fig = px.scatter_matrix(data_frame=df, title="Heatmap", width=2000, height=2000,
                       labels={col:col.replace('_', ' ') for col in df.columns})
fig.show()

Correlation

df.corr()

	team	targeted_productivity	smv	wip	over_time	incentive	idle_time	idle_men	no_of_style_change	no_of_workers	actual_productivity	prod_diff
team	1.00	0.03	-0.11	-0.01	-0.10	0.02	0.00	0.03	-0.01	-0.08	-0.15	-0.18
targeted_productivity	0.03	1.00	-0.07	0.06	-0.09	0.18	-0.06	-0.05	-0.21	-0.08	0.42	-0.15
smv	-0.11	-0.07	1.00	-0.48	0.74	0.64	0.06	0.11	0.32	0.91	-0.12	-0.09
wip	-0.01	0.06	-0.48	1.00	-0.29	-0.25	-0.11	-0.21	-0.26	-0.48	0.20	0.18
over_time	-0.10	-0.09	0.74	-0.29	1.00	0.58	0.05	-0.00	0.15	0.79	-0.06	-0.01
incentive	0.02	0.18	0.64	-0.25	0.58	1.00	-0.04	-0.07	0.06	0.72	0.23	0.14
idle_time	0.00	-0.06	0.06	-0.11	0.05	-0.04	1.00	0.56	-0.01	0.06	-0.08	-0.05
idle_men	0.03	-0.05	0.11	-0.21	-0.00	-0.07	0.56	1.00	0.13	0.11	-0.18	-0.17
no_of_style_change	-0.01	-0.21	0.32	-0.26	0.15	0.06	-0.01	0.13	1.00	0.33	-0.21	-0.10
no_of_workers	-0.08	-0.08	0.91	-0.48	0.79	0.72	0.06	0.11	0.33	1.00	-0.06	-0.01
actual_productivity	-0.15	0.42	-0.12	0.20	-0.06	0.23	-0.08	-0.18	-0.21	-0.06	1.00	0.83
prod_diff	-0.18	-0.15	-0.09	0.18	-0.01	0.14	-0.05	-0.17	-0.10	-0.01	0.83	1.00

pd.DataFrame(df.corr()["actual_productivity"].sort_values())

	actual_productivity
no_of_style_change	-0.21
idle_men	-0.18
team	-0.15
smv	-0.12
idle_time	-0.08
over_time	-0.06
no_of_workers	-0.06
wip	0.20
incentive	0.23
targeted_productivity	0.42
prod_diff	0.83
actual_productivity	1.00

plt.figure(figsize=(16,9))
sns.heatmap(df.corr(),cmap="coolwarm",annot=True,fmt='.2f',linewidths=2)
plt.title("", fontsize=20)
plt.show()

Answer These Questions

Why are these KPIs key to answering your business question or achieving your project goals? What specific insight did these KPIs or worksheets provide you?

These KPIs gave insights how these will affect workers productivity in terms of days, teams, incentives etc.

How does your dashboard or worksheet(s) exemplify use of at least one pre-attentive attribute?

The usage of colors to make distinctions or highlight important areas.

How does your dashboard or worksheet(s) exemplify use of at least one of the following KPI principles?

The graphs able to highlight main points in this dashboard

Do these worksheets meet the needs of your intended audience as-is? How might they need to be changed before ultimately being presented?

Yes, I made them as easy to understand as possible.

Python code done by Dennis Lam